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Abstract: In this note we prove the following law of the iterated logarithm for the Grenander 
estimator of a monotone decreasing density: If /(to) > 0? f'ifo) < 0, and f' is continuous in a 
neighborhood of to, then 

limsup ) {fn{to) -/(io)) = \f{to)f'{to)/2\^^^ 2M 

n-^oo V 2 log log n/ 

almost surely where 

M = sup Tg = (3/4)^/^ and Tg = argmax{ 5 '(u) — 
g^Q ^ 

here Q is the two-sided Strassen limit set on R. The proof relies on laws of the iterated logarithm 
for local empirical processes, Groeneboom’s switching relation, and properties of Strassen’s limit set 
analogous to distributional properties of Brownian motion. 
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1. Introduction: the MLE of a monotone density 


Nonparametric estimation of a monotone density was first considered by Grenander [1956]. Suppose that 
Xi,..., Xn are i.i.d. with distribution function F on [0, oo) having a decreasing density /. Grenander showed 
that the maximum likelihood estimator /„ of / is the (left-) derivative of the least concave majorant of the 
empirical distribution function F„ 

fn = {left derivative of the least concave majorant of F„}. 
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Fig 1. Empirical distribution and Least concave majorant, n = 10 



Fig 2. Grenander Estimator and Exp(l) density, n = 10 


The asymptotic distribution of fn{to) at a fixed point to with /^(to) < 0 was obtained by Prakasa Rao 
[1969], and given a somewhat different proof by Groeneboom [1985]. If /'(to) < 0 and /' is continuous in a 
neighborhood of to, then 


n^^^{fn{to) - /(to)) ■^f(,to)f{to) 


1/3 


2Z, 


( 1 . 1 ) 


where 


2Z = slope at 0 of the least concave majorant of W{t) — (1-2) 

= slope at 0 of the greatest convex minorant of W (t) + t^ 

= 2 argmin{Vh(t) + t^}; 

tGB 


here {W{t) : t G K} is a two-sided Brownian motion process starting at 0. In fact, the convergence in (1.1) 
can be extended to weak convergence of the (local) Grenander process as follows. Let {E>a,b{t) : t G K} denote 
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the slope process corresponding to the least concave majorant of Xa,b{t) = aW{t) — bt^, with a = \/f{to) 
and b = |/'(to)|/2. Then for fixed to with /'(to) < 0 and /' continuous in a neighborhood of to, 

n^^^{fn{to + n~^/'^t) - /(to)) ^ Sa,t)(t) 

in the Skorokhod topology on D[—K,K] for every finite K > see e.g. Groeneboom [1989], Kim and 
Pollard [1990], and Huang and Zhang [1994]. Groeneboom [1989] gives a complete analytic characterization 
of the limiting distribution Z and further, the distributional structure of the process S. The distribution 
of Z = S(0)/2 has been studied numerically by Groeneboom and Wellner [2001] which relies heavily on 
Groeneboom [1985] and Groeneboom [1989]. Balabdaoui and Wellner [2014] show that the distribution of 
Z is log-concave. Note that there is an “invariance principle” involved here: the centered slope of the least 
concave majorant of F„ converges weakly to a constant times the slope of the least concave majorant of 
X{t) = W{t) — t^. We can regard the slope in this Gaussian limit problem, 2Z, as an “estimator” of the 
slope of the line 2t in the Gaussian problem of “estimating” the “canonical” linear function 2t in “Gaussian 
white noise” dW{t) since 

dX{t) = 2tdt + dW{t). 


2. A law of the iterated logarithm for the Grenander estimator 

Our main goal is to prove the following Law of the Iterated Logarithm (LIL) for the Grenander estimator 
corresponding to the limiting distribution result in (1.1). 


Theorem 1. Suppose that /(to) > 0; foi^o) < 0 with f continuous in a neighborhood of to. Then 


almost surely where 


lim sup 

n—>-<50 


n^/^duitp) - /(to)) 

(2 log logn)^/3 


^/(io)/'(to) 


1/3 

2M 


M = supargmax{g(t) — t^} 
q^G 



here Q is the two-sided Strassen limit set on K given by 


g 


I git) = / 9 is)ds, t G 


g'^{s)ds < 1 


( 2 . 1 ) 


Our proof of Theorem 1 will rely on functional laws of the iterated logarithm for the local empirical process 
established by Mason [1988]; see also Deheuvels and Mason [1994], Einmahl and Mason [1998], Einmahl and 
Mason [1997], and Mason [2004]. Along the way we will also prove several lemmas concerning the limit set 

g. 


Proof. We begin the proof of Theorem 1 with a switching argument. Let = (n ^2 log log n)^/^. Then we 
want to find a number xo such that 


PiK^ifnito) - /(to)) > X i.o.) 


0, if x > Xo, 

1, if X < Xq. 
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Now we let 


s"„(a) = argmax{F„(s) — as}, a > 0, 

S 


( 2 . 2 ) 


and note that {fn{to) > a} = {s„(a) > to} by Groeneboom’s switching relation (see e.g. Groeneboom [1985], 
van der Vaart and Wellner [1996] page 296, and Balabdaoui et al. [2011], Theorem 2.1, page 881). Thus the 
event in the last display can be rewritten as 

\^fnito) > fito)+ bnX i.o.j = {sn{f (to) + b^x) > to i.o.} . (2.3) 

But, by letting s = to + bnh in (2.2) we see that 

Snifito) + bnx) - to = argmax{F„(to + 6„ti) - (/(to) + 6„a;)(to + 6nti)}, 

h 

and hence the right side of (2.3) can be rewritten as {/i„ > 0 i.o.} where 

K = argmax{F„(to + 5„/i) - (/(to) + 5„a;)(to + 6„ti)} 

h 

= argmax {5“^{F„(to + bnh) - F„(to) - {F{to + bnh) - F{to))} 

h 

+ 5“^{F(to + bnh) - F{to) - f{to)bnh} - xh} . (2.4) 

The second term on the right side in the last display converges to /'(to)/i^/2 as n —)■ oo. The handle the first 
term we appeal to (a slight extension of) Theorem 2 of Mason [1988]; see also Deheuvels and Mason [1994] 
Theorem A and Theorem 1.1, pages 1620-1621: by considering ti G K and introducing the two-sided version 
Q of the Strassen limit set given in (2.1) much as in Wichura [1974], we see that the sequence of functions 

{6/^{F„(to -f bnh) - F„(to) - (F(to -f bnh) - F{to))} : he R} 

is almost surely relatively compact with limit set 

{gif (to)-) ■ 9 &G} 


where G is given by (2.1). 

This is most easily seen as follows: let (G„ be the empirical d.f. of i.i.d. Uniform(0,1). As 

in Deheuvels and Mason [1994], with so that (2 log log n)^/^ ^ oo and 

'n~^kn = bn \ 0, the processes 

fnjs) 

V2 log logn 

_ {GniFjto + n-^kns)) - GnjFjto)) - {F{to + n~^fcns) - F{to))} 

y/kn/n ^2 log logn 

with s > 0 are almost surely relatively compact with limit set /Coo(c) = {t i—>■ g{ct) : g G /Coo} with c = /(to)- 
Here we also note that 


y^kn/ny/21og logn (2 log logn)2/3 
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Thus the processes involved in the argmax in (2.4) are almost surely relatively compact with limit set 

{9{f{to)h)+ 2~^f\to)h'^ - xh : g G G}, 
and by Lemma 1 below this set is equal to 

{ag{h) — bh^ — xh : g G G} 

where a = sj /(to), and b = |/'(to)|/2. Thus by Lemma 2 below, the set of limits for the argmax in (2.4) 
equals 

(a/6)^/^ argmax{ 5 (/i) — /i^} — xj{2b) : g G G 

h 


where 


id 


4/(to) 


2-^l/'(to)| / Vl/'(^o)| 


Hence, with Tg = aigmaxf^{g{h) — /i^}. 


{X„>0i,o,} “=■■ {(^)’''»pr,>^} 


if 


/ \ 2/3 

a;>a;o = 26 (-) supTg = -f{to)f'{to) 

Vo/ ggg 2 

It remains only to show that sup^gg Tg = (3/4)This follows from Lemma 3 in Section 4 below. □ 

Lemma 1. Let c > 0 and d gM.. Then 

{t !->• g{ct + d) — g{d) : g G G} = \fcQ- 

Proof. It g G G, then 

pCt-\-d net nt 

g{ct + d) — g{d) = / g{s)ds = / g{v + d)dv = / g{cu + d)cdu 

Jd Jo Jo 

= \fc / \fcg{cu + d)du 

Jo 


1/3 


2 sup Tg . 
geO 


= Vcg{t) 


where g G G since 


/ oo noo 

{y/cg{cu + d))^du = / g^{w)dw<l. 

-OO —OO 


This shows that the set of functions t i—>■ g{ct + d) — g{d), g G G, ts contained in y^cG- On the other 
hand, any function g G G with derivative g may be written as g{t) = J y^cg(cu + d)du with g given by 
g{s) = Vc~^gic~^s — c~^d) and satisfying g{.s)‘^ds = g(syds <1. □ 
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Lemma 2. Let a,j3 be positive constants and 7 € M. Then 

|argmax{Q;(7(/i) — — 7/1} : 5 e 

= |(a//3)^^^argmax{6f(ft,) -/i^} - 7/(2/3) : g G ■ 

Proof. Note first that 


Mg = argmax |ag(h) — fdh? — 7/1} 

h 

= argmax {ag(h) — ^(/i + 7 /( 2 /?))^} 

h 

= argmax - (/3/a)(h + 7 /( 2 /?)) 2 } 

h 

= argmax {g(r; + d) — (/3 /q;)v^} + c? 


with d := — 7 /( 2 /?). Moreover, for any c > 0 and 

g{u) = {g{cu + d) - g{d)) 


we may write 

Mg = ca,Tguia.x{g{cu +d) — g{d) — {/3/a)c^u^} + d 

U 

= cargmax — (/3/a)c^M^ I + d 

U ^ ' 

= c argmax |5 (m) — (/3/q;)c^^^'u^| + d. 


In case of c = (a//3)^/^ we obtain 

Mg = argmax { 5 (m) - u^} - 7 /( 2 /?). 


Now the claim follows from Lemma 1, because the set {g ■ g G G} equals G- 


3. Some comparisons and connections 

As noted in the introduction, 

2Z = slope at zero of the least concave majorant of W{t) — t^. 

This suggests that with Tg = argmaX({g(t) — we have 

{ 2 suprg : g gG} 

= supjslope at 0 of the least concave majorant of g{t) — t^ : g G G}- 


(2.5) 


□ 
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4. Proof for the variational problem 


It is natural to conjecture that sup^gg Tg = (3/4)^/^ « 0.90856_ This is motivated by the asymptotic 

behavior of Chernoff’s density; see Groeneboom [1989], Corollary 3.4, page 94: since the density 




1 


as z 


2Ai'{ai) 

oo, the tail probability P{Z > z) satisfies 


44/3 


zexp 


3^ 


,31/3 


aiz 


P(Z > z) 


_ ^ _44/3 1 

2Ai'{ax) z 


exp 



as z —)■ 00 where ai= — 2.3381 is the largest zero of the Airy function Ai and Ai'(ai)=0.7022. Thus from 
(1.1) we expect that 


lim sup 


- /(to)) 

((3/2) log log n) 4/3 





or, equivalently. 


lim sup 

n—>-<50 


- /(to)) 
(2 log log n)4/3 


^/(/o)/'(/o) 

^/(/o)/'(/o) 


1/3 1 

2 . _ . 

21/3 



1/3 



On the other hand the proof of Theorem 1 above leads to 


lim sup 

n—>oo 


- /(to)) 

(2 log log n) 1/3 




1/3 

2 • M 


a.s. 


where 

M = supargmaxtgjj{5(/) - = sup Tg. 

geO geG 


Thus we conjecture that M = (3/4) i/3. 


Lemma 3. Let tQ > 0 be an arbitrary positive number and let g G Li([0,to]) be an arbitrary function 
satisfying 



g{s)ds -tl> 


f g(s)ds-t^ 
Jo 


for 0 <t <to. 


Then 


rto pto Af3 

g{updu > / {2u)^du = —2.. 


Proof. Let go{u) = ‘2,u. The claimed inequality is trivial if the integral on the left side is infinite, so we may 
view g and go as elements of the Hilbert space L2([0,to])- Then the assumption on g may be rewritten as 

( 5 - 50 , 1 ) > ( 5 - 50 , l[o,t]) for 0<t<to. 
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In other words, 


and this is equivalent to 


(5-5o,l(t.io]) > 0 forO<t<to, 


(g-goj) > 0 


for all functions / in the closed convex cone K generated by the indicator functions This is the set of 

non-negative and non-decreasing functions on [0,to]- In particular, go G K, so 

{g- k,go) > 0 . 


Together with the Cauchy-Schwarz inequality we obtain 


0 < (5 - go,go) = {g,go) - llsof < \\g\\\\go\\ - llsof , 


so ll^ll > ll^oll- This inequality is strict unless g = Ago for some A € M. In this special case the last display 
reads 0 < (A — l)||go|P, so A > 1 and IjgH = A||go|| with equality if, and only if, A = 1 and g = go- □ 

Example 1. If we take f{x) = e“®l[o,oo)(a^) and to = log 2, then 


2 /(fo)/'(fo) 


1/3 


. 2 = - 2 = 1 , 


so the limit superior is just sup^gg Tg = (3/4)^/^. 

Example 2. If we take f(x) = (1 -|- a:)“^l[o oo)(x), then —f{x) = 2(1 -|- x)~^ and hence with to = I we have 
f{l) = l/A = -fil). Then 


^fito)f'ito) 


1/3 


2 = ( 2 “®/^) • 2 = 2 “^/^, 


so the limit superior is 2 sup^gg Tg = (3/16)^/^. 

Example 3. If we take f{x) = (-\/2 — a;)ljo y 2 ](^) fo = ~ /(fo) = 1> —f'{to) = I; and 


lf{to)f{to)\'^^ ■ 2 = ( 2 - 1 / 3 ) . 2 = 2 + 2 / 3 , 


so the limit superior is 2+^/3 sup^gg Tg = 22 / 3 ( 3 / 4 ) 1/3 = 31 / 2 , 


5. Some corollaries 

Theorem 1 has a number of corollaries and consequences, since the argument in the proof applies to a number 
of problems involving nonparametric estimation of a monotone function. Our first corollary, however, involves 
estimation of the mixing distribution G in the mixture representation of a monotone density: that is, 

/.oo 1 /■ 1 

f{x)= -l[oy){x)dG{y)= -dG{y), x G (0, 00 ) (5.1) 

Jo y J{y>x} y 
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for some distribution function G on (0,oo). This fact apparently goes back at least to Schoenberg [1941]; see 
the introduction of Williamson [1956], and Feller [1971], page 158. The relationship (5.1) implies that the 
corresponding distribution function F is given by 

F{x) = / -l[o,y)(x)dG(y) + / l[y^^){x)dG{y) 

Jo y Jo 

= xf{x) + G{x ), 
and this can be “inverted” to yield 

G{x) = F{x) — xf{x). (5.2) 

From Figure 3 we see that the function on the right side of (5.2) is non-negative and non-decreasing: the 
shaded area gives exactly the difference F{x) — xf{x). 



Fig 3. Graphical view of the inversion formula, monontone density 


The identity (5.2) implies that the nonparametric maximum likelihood estimator of G is Gn given by 


Gn{t) = Fn{t) for t>0 

where Fn{t) = f f„(x)dx is the least concave majorant of F„ and the MLE of F assuming that / is monotone 
(and hence F is concave). Thus for to > 0 we can write 

n^^^(Gn(to) - G(to)) = n^/^(F„(to) - F(to)) - ton^^^{fn{to) - /(to)) 


From Marshall’s lemma Marshall [1970] and n^/^llF^-FJloo = Op(l) it follows that -Fjjoo = Op(l). 

Thus if to > 0 is a point at which the hypotheses of Theorem 1 hold, then the convergence in (1.1) implies 


that 


n^/^(G„(to) - G(to)) -^d to ^f{to)f{to) 


1/3 


2Z, 


(5.3) 


Similarly, from Marshall’s lemma Marshall [1970] and Chung’s law of the iterated logarithm for j|Fji — Fjjoo 
(see e.g. Shorack and Wellner [1986], page 505), we know that with = (2 log log n)^/^ 


limsupn^/^||F„ - F\\^/bn < limsupn^/^||F„ - F\\^/bn = 1/2 a.s.. 

n—yoo n—>-oo 


It follows that if to > 0 is a point at which the hypotheses of Theorem 1 hold, then Theorem 1 yields a LIT 
result for G„(to) as follows: 
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Collorary 1. Suppose that /(io) > 0 and f'{to) < 0 with j' continuous in a neighborhood of to- Then 


lim sup 

n—XX) 


- Gjto)) 

(2 log log n) 


to 


^f{to)fito) 


1/3 


2(3/4) 


1/3 


almost surely. 


6. A further problem 

For the problem of estimating a convex decreasing density, Groeneboom, Jongbloed and Wellner [2001] 
described the limiting distribution of the estimator (at a point under a natural curvature condition) in terms 
of an “invelope” of two-sided integrated Brownian motion plus which was characterized in Groeneboom, 
Jongbloed and Wellner [2001]. The same distribution has appeared in other nonparametric convex function 
estimation problems, for example for log-concave density estimation: see Balabdaoui, Rufibach and Wellner 
[2009] . In spite of this description of the limiting distribution for the convex density case in terms of integrated 
Brownian motion, almost nothing is known concerning a direct analytical description of the limit distribution 
comparable to the results of Groeneboom [1985, 1989] for Ghernoff’s distribution. (On the other hand, a 
preliminary numerical investigation of the distribution is given by Azadbakhsh, Jankowski and Gao [2014].) 

This leads to the following question: can some information concerning the constants involved in the 
limiting distribution in the convex function case be obtained by establishing LIT results analogous to those 
established here in the monotone case? 


Acknowledgements 

The second author owes thanks to Piet Groeneboom for many discussions concerning the Grenander estimator 
and monotone function estimation more generally. Thanks are due as well to David Mason for references 
concerning local LIL’s for empirical processes. 


References 

Azadbakhsh, M., Jankowski, H. and Gao, X. (2014). Computing confidence intervals for log-concave 
densities. Comput. Statist. Data Anal. 75 248-264. MR3178372 
Balabdaoui, F., Rufibach, K. and Wellner, J. A. (2009). Limit distribution theory for maximum 
likelihood estimation of a log-concave density. Ann. Statist. 37 1299-1331. MR2509075 (2010h:62290) 
Balabdaoui, F. and Wellner, J. A. (2014). Chernoff’s density is log-concave. Bernoulli 20 231-244. 
MR3160580 

Balabdaoui, F., Jankowski, H., Pavlides, M., Seregin, A. and Wellner, J. (2011). On the Grenander 
estimator at zero. Statist. Sinica 21 873-899. MR2829859 
Deheuvels, P. and Mason, D. M. (1994). Functional laws of the iterated logarithm for local empirical 
processes indexed by sets. Ann. Probab. 22 1619-1661. MR1303659 (96e:60048) 

Einmahl, U. and Mason, D. M. (1997). Gaussian approximation of local empirical processes indexed by 
functions. Probab. Theory Related Fields 107 283-311. MR1440134 (98d:60060) 

Einmahl, U. and Mason, D. M. (1998). Strong approximations to the local empirical process. In High 
dimensional probability (Oberwolfach, 1996). Progr. Probab. 43 75-92. Birkhauser, Basel. MR1652321 
(99h:60060) 

Feller, W. (1971). An introduction to probability theory and its applications. Vol. II. Second edition. John 
Wiley & Sons, Inc., New York-London-Sydney. MR0270403 (42 


imsart-generic ver. 2012/08/31 file: LIL-grenander-v8.tex date: February 3, 2015 





Diimbgen, Wellner, and Wolff/LIL for Grenander 


11 


Grenander, U. (1956). On the theory of mortality measurement. I. Skand. Aktuarietidskr. 39 70-96. 
MR0086459 (19,188d) 

Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley conference in 
honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983). Wadsworth Statist./Probab. Ser. 
539-555. Wadsworth, Belmont, CA. MR822052 (871:62076) 

Groeneboom, P. (1989). Brownian motion with a parabolic drift and Airy functions. Probab. Theory 
Related Fields 81 79-109. MR981568 (90c:60052) 

Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001). A canonical process for estimation 
of convex functions: the “invelope” of integrated Brownian motion +t^. Ann. Statist. 29 1620-1652. 
MR1891741 (2003c:62075) 

Groeneboom, P. and Wellner, J. A. (2001). Computing Chernoff’s distribution. J. Comput. Graph. 
Statist. 10 388-400. MR1939706 

Huang, Y. and Zhang, C.-H. (1994). Estimating a monotone density from censored observations. Ann. 
Statist. 22 1256-1274. MR1311975 (95m:62085) 

Kim, j. and Pollard, D. (1990). Cube root asymptotics. Ann. Statist. 18 191-219. MR1041391 (91f:62059) 
Marshall, A. W. (1970). Discussion on Barlow and van Zwet’s paper. In Nonparametric Techniques in 
Statistical Inference, (M. L. Puri, ed.). Proceedings of the First International Symposium on Nonparametric 
Techniques held at Indiana University, June 1969 174-176. Cambridge University Press, London. 
MR0273755 (42 ##8632) 

Mason, D. M. (1988). A strong invariance theorem for the tail empirical process. Ann. Inst. H. Poincare 
Probab. Statist. 24 491-506. MR978022 (89m:60076) 

Mason, D. M. (2004). A uniform functional law of the logarithm for the local empirical process. Ann. 
Probab. 32 1391-1418. MR2060302 (2005f:60080) 

Prakasa Rao, B. L. S. (1969). Estimation of a unimodal density. Sankhyd Ser. A 31 23-36. MR0267677 
(42 ##2579) 

Schoenberg, I. J. (1941). On integral representations of completely monotone and related functions 
(abstract). Bull. Amer. Math. Soc. 47 208. 

Shorack, G. R. and Wellner, J. A. (1986). Empirical processes with applications to statistics. Wiley 
Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & 
Sons, Inc., New York. MR838963 (88e:60002) 

VAN DER Vaart, A. W. and Wellner, J. A. (1996). Weak convergence and empirical processes. Springer 
Series in Statistics. Springer-Verlag, New York With applications to statistics. MR1385671 (97g:60035) 
WiCHURA, M. J. (1974). Functional laws of the iterated logarithm for the partial sums of LED. random 
variables in the domain of attraction of a completely asymmetric stable law. Ann. Probability 2 1108-1138. 
MR0358950 (50 ##11407) 

Williamson, R. E. (1956). Multiply monotone functions and their Laplace transforms. Duke Math. J. 23 
189-207. MR0077581 (17,1061d) 


imsart-generic ver. 2012/08/31 file: LIL-grenander-v8.tex date: February 3, 2015 


